Skip to content

Create tests/distributed/test_mnnvl_alltoall.py#35241

Open
puririshi98 wants to merge 47 commits intovllm-project:mainfrom
puririshi98:patch-1
Open

Create tests/distributed/test_mnnvl_alltoall.py#35241
puririshi98 wants to merge 47 commits intovllm-project:mainfrom
puririshi98:patch-1

Conversation

@puririshi98
Copy link
Copy Markdown

@puririshi98 puririshi98 commented Feb 24, 2026 •
edited
Loading

all 5 tests pass on 8xh100 w/ latest nvidia stack

This is part of the NVIDIA effort to add CI to upstream github

GPU Hours Estimate:

  • ~0.3 GPU hours
  • 8 tests requiring 2-4 GPUs
  • Wall-clock: ~5 minutes

@puririshi98
Create tests/distributed/test_mnnvl_alltoall.py
a93eb3d 
all 5 tests pass on 8xh100 w/ latest nvidia stack 

Signed-off-by: Rishi Puri <riship@nvidia.com>
gemini-code-assist[bot]
gemini-code-assist bot reviewed Feb 24, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request introduces a new test file for MNNVL AllToAll operations, ensuring the correct functionality and initialization of FlashInfer components within a distributed environment. The tests cover manager initialization, workspace reinitialization, and the ensure_initialized method, as well as a custom communicator wrapper. The setup correctly handles multi-GPU environments and checks for necessary system capabilities like SYS_PTRACE.

One area for improvement is the broad exception handling in has_sys_ptrace_capability, which could mask underlying issues.

tests/distributed/test_mnnvl_alltoall.py Outdated
Comment on lines +60 to +61
except Exception:
pass
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Catching a generic Exception can hide specific issues that might arise during the file reading or parsing of /proc/self/status. It's generally better to catch more specific exceptions (e.g., IOError, ValueError) to avoid masking other potential bugs. While this function is for capability checking, a more precise exception handling would improve maintainability and debugging.

Suggested change
except Exception:
pass
except (IOError, ValueError) as e:
# Log the error for debugging purposes, but continue with alternative checks
print(f"Warning: Error reading /proc/self/status: {e}")

@mergify
Copy link
Copy Markdown

mergify bot commented Feb 24, 2026

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?
mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

@mergify
Copy link
Copy Markdown

mergify bot commented Feb 25, 2026

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?
mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

@mergify
Copy link
Copy Markdown

mergify bot commented Feb 26, 2026

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?
mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

@puririshi98
fix lint
a0bb03a 
Signed-off-by: Rishi Puri <riship@nvidia.com>
@mergify
Copy link
Copy Markdown

mergify bot commented Feb 27, 2026

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?
mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

@mergify
Copy link
Copy Markdown

mergify bot commented Feb 27, 2026

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?
mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

@mergify
Copy link
Copy Markdown

mergify bot commented Feb 27, 2026

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?
mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

@jasonlizhengjian jasonlizhengjian mentioned this pull request Mar 6, 2026
Open
@puririshi98
Update distributed.yaml
c9cb5f8 
Signed-off-by: Rishi Puri <riship@nvidia.com>
@mergify mergify bot added the ci/build label Mar 6, 2026
jasonlizhengjian
jasonlizhengjian reviewed Mar 6, 2026
View reviewed changes
puririshi98 and others added 2 commits March 6, 2026 14:47
@puririshi98
Copy link
Copy Markdown
Author

puririshi98 commented Mar 9, 2026 •
edited
Loading

recent changes:

tests/distributed/test_mnnvl_alltoall.py::test_flashinfer_all2all_import PASSED                                                                                                                                                              [ 12%]
tests/distributed/test_mnnvl_alltoall.py::test_flashinfer_alltoall_manager_initialization[2] PASSED                                                                                                                                          [ 25%]
tests/distributed/test_mnnvl_alltoall.py::test_flashinfer_alltoall_workspace_reinitialization[2] PASSED                                                                                                                                      [ 37%]
tests/distributed/test_mnnvl_alltoall.py::test_flashinfer_alltoall_ensure_initialized[2] PASSED                                                                                                                                              [ 50%]
tests/distributed/test_mnnvl_alltoall.py::test_alltoall_data_communication[2] PASSED                                                                                                                                                         [ 62%]
tests/distributed/test_mnnvl_alltoall.py::test_flashinfer_alltoall_data_communication[2] PASSED                                                                                                                                              [ 75%]
tests/distributed/test_mnnvl_alltoall.py::test_alltoall_deterministic_data_validation[2] PASSED                                                                                                                                              [ 87%]
tests/distributed/test_mnnvl_alltoall.py::test_custom_communicator PASSED

on dgxh100

puririshi98 and others added 2 commits March 9, 2026 16:24
@puririshi98
Update test_mnnvl_alltoall.py
ea9f002 
Signed-off-by: Rishi Puri <riship@nvidia.com>
@puririshi98
Merge branch 'main' into patch-1
e95863c
@puririshi98
Copy link
Copy Markdown
Author

puririshi98 commented Mar 10, 2026

Can you confirm?

Yes, this test refers to the MNNVL all2allv implementation from PR #21003.

The test file validates FlashInferAllToAllManager which:

  • Uses flashinfer.comm.trtllm_alltoall.MnnvlMoe APIs
  • Is configured via VLLM_ALL2ALL_BACKEND=flashinfer_all2allv

@mergify
Copy link
Copy Markdown

mergify bot commented Mar 11, 2026

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

@mergify
Copy link
Copy Markdown

mergify bot commented Mar 11, 2026

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

@mergify
Copy link
Copy Markdown

mergify bot commented Mar 11, 2026

Hi @puririshi98, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

puririshi98 and others added 6 commits March 11, 2026 18:51
@puririshi98
precommit
3afdc0c 
Signed-off-by: Rishi Puri <riship@nvidia.com>
@puririshi98
Merge branch 'patch-1' of https://github.com/puririshi98/vllm into pa…
f54a6b4 
…tch-1
@puririshi98
Update distributed.yaml
0e5a46c 
Signed-off-by: Rishi Puri <riship@nvidia.com>
@puririshi98
Merge branch 'main' into patch-1
f889832
@puririshi98
Merge branch 'main' into patch-1
c4b7df0
@claude
Fix pre-commit: Replace torch.cuda API calls with torch.accelerator
d76d7ac 
Replace deprecated torch.cuda API calls with torch.accelerator equivalents:
- torch.cuda.set_device() → torch.accelerator.set_device_index()
- torch.cuda.current_device() → torch.accelerator.current_device_index()
- torch.cuda.device_count() → torch.accelerator.device_count()

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

Signed-off-by: Claude <claude@anthropic.com>
@hjjq
Copy link
Copy Markdown
Contributor

hjjq commented Mar 20, 2026

How hard would it be to add flashinfer_moe_a2a (flashinfer_nvlink_one_sided in current main branch) to this test?
There was also renaming in this PR: #36022
flashinfer_all2allv -> flashinfer_nvlink_two_sided
flashinfer_moe_a2a -> flashinfer_nvlink_one_sided

Might be better to make the names consistent in this test for clarity?

@puririshi98
Copy link
Copy Markdown
Author

puririshi98 commented Mar 20, 2026

How hard would it be to add flashinfer_moe_a2a (flashinfer_nvlink_one_sided in current main branch) to this test? There was also renaming in this PR: #36022 flashinfer_all2allv -> flashinfer_nvlink_two_sided flashinfer_moe_a2a -> flashinfer_nvlink_one_sided

Might be better to make the names consistent in this test for clarity?

WIP

hjjq
hjjq reviewed Apr 1, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@hjjq hjjq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @puririshi98 I've left some questions. PTAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

4 more reviewers

@jasonlizhengjian jasonlizhengjian jasonlizhengjian left review comments

@hjjq hjjq hjjq left review comments

@wzhao18 wzhao18 wzhao18 left review comments

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

ci/build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@puririshi98 @jasonlizhengjian @leo-cf-tian @hjjq @wzhao18 @claude